Rotorcraft collision avoidance system and related method

ABSTRACT

A method of operating a rotorcraft collision avoidance system is provided. The method includes determining a unique characteristic of a detected rotorcraft, determining the actual length of a first rotor based at least in part on the determined unique characteristic of the rotorcraft, locating a major axis of the first rotor of the rotorcraft from a perspective of a home unit, from the perspective of the home unit, determining an angular extent of the major axis of the first rotor, and determining the then current distance from the home unit to the rotorcraft based at least in part on the determined actual length of the first rotor and the corresponding angular extent.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.14/162,476, filed Jan. 23, 2014, pending, which will issue as U.S. Pat.No. 9,177,482, on Nov. 3, 2015, the disclosure of which is herebyincorporated herein in its entirety by this reference.

BACKGROUND

Currently, more military helicopters are lost each year to mid-aircollisions with other helicopters than are downed by enemy fire. Whilethe problem is not newly discovered, there has been little effortexpended in finding practical solutions for military rotary wingaircraft, particularly, solutions that do not require additionalequipment/systems to be installed on the platform (due to size, weightand power constraints).

For the reasons stated above and for other reasons stated below, whichwill become apparent to those skilled in the art upon reading andunderstanding the present specification, there is a need in the art foran effective and efficient helicopter collision avoidance system.

BRIEF SUMMARY

The above-mentioned problems of current systems are addressed byembodiments of the present invention and will be understood by readingand studying the following specification. The following summary is madeby way of example and not by way of limitation. It is merely provided toaid the reader in understanding some of the aspects of the invention.

In one embodiment, a method of operating a rotorcraft collisionavoidance system is provided. The method includes: determining a uniquecharacteristic of a detected rotorcraft; determining the actual lengthof a first rotor based at least in part on the determined uniquecharacteristic of the rotorcraft; locating a major axis of the firstrotor of the detected rotorcraft from a perspective of a home unit; fromthe perspective of the home unit, determining an angular extent of themajor axis of the first rotor; and determining the current distance fromthe home unit to the detected rotorcraft based, at least in part, on thedetermined actual length of the first rotor and the correspondingangular extent of the major axis of the first rotor.

In still another embodiment, a method of operating a rotorcraftdetection system is provided. The method includes: taking a first set ofimages of at least one rotor of a rotorcraft over a first period of timewith a sensor of a home unit; temporally filtering the first set ofimages; determining the frequency of the at least one rotor based on thetemporally filtered first set of images; matching the determinedfrequency of the at least one rotor with a frequency of a knownrotorcraft; determining an actual diameter of the at least one rotorbased on the match; locating a major axis in the temporally filteredfirst set of images; determining an angular extent between each end ofthe major axis; and determining the then current distance between thesensor of the home unit and the rotorcraft based on the determinedangular extent and the actual diameter of the at least one rotor.

In yet another embodiment, a rotorcraft collision avoidance system isprovided. The rotorcraft collision avoidance system includes at leastone imaging sensor, a controller, a memory and a communication device.The at least one imaging sensor is configured and arranged to captureimages of nearby rotorcrafts. The controller is in communication withthe at least one imaging sensor to receive the images from the at leastone imaging sensor. The controller is further configured to process thecaptured images to determine distances to the rotorcrafts based on atleast identifying the rotorcraft with the images, identifying a majoraxis of a main rotor of the rotorcraft with the images and determiningan angular extent between each end of the major axis. The memory is incommunication with the controller to store processed image data andinstructions. In addition, the communication device is in communicationwith the controller. The communication device is configured and arrangedto communicate the determined distance upon direction of the controller.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention can be more easily understood and furtheradvantages and uses thereof will be more readily apparent, whenconsidered in view of the detailed description and the following figuresin which:

FIG. 1 is a collision avoidance flow diagram of one embodiment of thepresent invention;

FIG. 2 is a block diagram of a collision avoidance system of oneembodiment that would implement the collision avoidance flow diagram ofFIG. 1;

FIG. 3A is a characteristic determination flow diagram of one embodimentof the present invention;

FIG. 3B is another characteristic determination flow diagram of anotherembodiment of the present invention;

FIG. 4A is a temporal graph example illustrating a temporal signal levelof a single pixel over a period of 640 ms;

FIG. 4B is a spectral signature graph of the corresponding pixel in thetemporal graph of FIG. 4A;

FIG. 5 is an angular extent flow diagram of an embodiment of the presentinvention;

FIG. 6A is an example ellipse with a major axis and sensor of oneembodiment of the present invention; and

FIG. 6B is an illustration of vectors used to determine an angularextent in an embodiment of the present invention.

In accordance with common practice, the various described features arenot drawn to scale but are drawn to emphasize specific features relevantto the present invention. Reference characters denote like elementsthroughout the figures and the specification.

DETAILED DESCRIPTION

In the following detailed description, reference is made to theaccompanying drawings, which form a part hereof and in which is shown byway of illustration, specific embodiments in which the inventions may bepracticed. These embodiments are described in sufficient detail toenable those skilled in the art to practice the invention, and it is tobe understood that other embodiments may be utilized and that changesmay he made without departing from the spirit and scope of the presentinvention. The following detailed description is, therefore, not to betaken in a limiting sense, and the scope of the present invention isdefined only by the claims and equivalents thereof.

Embodiments of the present invention provide a system and method todetermine the location of a rotorcraft (a vehicle that uses a prop forflight such as, but not limited to, helicopters, planes with propellersor any other type of craft (vehicle) that uses a rotor (propeller) formovement) in relation to a position (home unit) to help prevent crashes.Embodiments determine the type of aircraft and length of the rotor(diameter of the rotor) with the use of one or more sensors. Embodimentsfurther determine, from a perspective of a home unit, a subtended anglecorresponding to the diameter of the rotor. From this data, adetermination of the location of the rotorcraft in relation to a homeunit position is determined. If a subsequent determination of thelocation of the rotorcraft indicates that the rotorcraft is encroaching,evasive maneuvers can be employed to prevent a crash. Moreover, inembodiments, when it is determined that a rotorcraft is on a collisioncourse (or even a near miss); it alerts a pilot and provides a bearingangle to the potential threat. Some embodiments implement sensorsalready used with the rotorcraft to make a distance determination. Forexample, most military helicopters are equipped with defensive systemsthat are capable of detecting when the helicopters are being engaged bynon-friendly sources by using thermal imaging sensors. An example ofsuch a defense system is the AN/AAR-59 Joint and Allied Threat AwarenessSystem (JATAS) from Alliant Techsystems Inc. Embodiments extend baselinecapabilities of the defense system to include early recognition ofprojected trajectory intersections with neighboring helicopters andalert the aircrew to potential mid-air collisions. A benefit to theembodiments that incorporate existing sensing equipment is that thesystem is unlikely to encounter additional issues typically associatedwith introducing a new system onto a military helicopter.

Referring to FIG. 1, a collision avoidance flow diagram 100 of oneembodiment is illustrated. The collision avoidance flow diagram 100provides an overview of an application of a collision avoidance methodwith a collision avoidance system of an embodiment. An example blockdiagram collision avoidance system 200 is illustrated in FIG. 2. As FIG.1 illustrates, the system starts by first detecting a presence of arotorcraft (102). In one embodiment, this would be done with a sensor,such as sensor 212 of the collision avoidance system 200 of FIG. 2. Anexample sensor 212 is a Mid-Wave Infrared (MWIR) imaging sensor, whichis capable of capturing images even in the dark and even in weatherconditions such as, but not limited to, clouds, rain, hail, dust, etc.Other types of sensors can be used to detect the presence of nearbyrotorcraft such as, but not limited to, radar systems, acoustic systems,etc. Once a rotorcraft 250 has been detected (102), the process as setout in FIG. 1 determines the actual length of a main rotor of therotorcraft 250 (which is further described in detail below) (104) anddetermines the then current angular extent to the main rotor of therotorcraft 250 (which is also further described in detail below) (106).Knowing the actual length of the main rotor and the measured angularextent, the distance to the rotorcraft 250 can be determined (thisdetermination is also further described in detail below) (108). In theembodiment of FIG. 2, a controller 204 that is in communication with thesensor 212 determines the distance based on instructions stored inmemory 210.

Once the distance is determined, the distance is communicated (110) asillustrated in FIG. 1. In the collision avoidance system 200 of FIG. 2,the controller 204 is configured to direct a communication device 208 tocommunicate the distance information. The communication device 208 maybe a display, a speaker, a combination of both, or any other type ofdevice that can convey a message about the distance to a user. Thedistance is the distance between the sensor 212 of a home unit 202 andthe rotorcraft 250. The home unit 202 may be another rotorcraft, such asanother helicopter, another type of aircraft, vehicle or even astationary structure. In the example embodiment of FIG. 1, if it isdetermined the distance is within a safety space of the home unit 202,an alarm is activated. The alarm may be implemented by the communicationdevice 208. The alarm provides the home unit 202 with a warning thatevasive actions are needed immediately to avoid a collision. As thecollision avoidance flow diagram 100 illustrates, this embodiment thendetermines if the then current distance is the first distance determinedby the collision avoidance system 200 for the rotorcraft (116). If it isthe first determined distance for the rotorcraft, the process continuesat (102). Each time a distance is determined it is stored in memory 210of the collision avoidance system 200. If it is determined (116) that itis not the first distance to be determined, the controller 204 uses datarelating to the prior determined distance and data relating to the thencurrent distance to determine the then current direction of travel ofthe rotorcraft (118) in relation to the home unit 202. In particular,this is done in an embodiment with knowledge of the rotor diameter, theangle of extent of the major axis to the main rotor of the rotorcraftand a pointing direction. With this information, an estimated locationof the rotorcraft in relation to the home unit at a specific time can bedetermined. By comparing the estimated location of the rotorcraft at twodifferent times the direction of travel is determined. The direction oftravel is then communicated via the communication device 208 (120). Asillustrated in FIG. 1, the process then continues at step 102, so thedistance and direction of the rotorcraft 250 is monitored until it isout of range of the sensor 212. Timing of the collection of data, suchas with images, by the sensor and the operation of the controller 204 isprovided by a clock 206, as illustrated in FIG. 2.

The methods and techniques described herein are implemented by one ormore computer systems such as the computer system employed in thecollision avoidance system 200 of FIG. 2. Embodiments of the computersystems may be implemented in digital electronic circuitry, or with aprogrammable processor (for example, a special-purpose processor or ageneral-purpose processor such as a computer), firmware, software, orcombinations of the above. Apparatus embodying these techniques mayinclude appropriate input and output devices, a programmable processor,and a storage medium tangibly embodying program instructions forexecution by the programmable processor. A process embodying thesetechniques may be performed by a programmable processor configured toexecute a program of instructions in order to perform desired functionsby operating on input data and generating appropriate output data. Thetechniques may advantageously he implemented in one or more programsthat are executable on a programmable system including at least oneprogrammable processor coupled to receive data and instructions from,and to transmit data and instructions to, a data storage system, atleast one input device, and at least one output device. Generally, aprocessor will receive instructions and data from a read-only memory(ROM) and/or a random access memory (RAM). Storage devices suitable fortangibly embodying computer program instructions and data include allforms of memory including, by way of example, non-volatile memory,semiconductor memory, such as erasable programmable read-only memory(EPROM), electrically erasable programmable read-only memory (EEPROM),and Flash memory; magnetic disks such as internal hard disks andremovable disks; magneto-optical disks; and CD-ROM discs. Any of theforegoing may be supplemented by, or incorporated in, specially designedapplication-specific integrated circuits (ASICs).

The actual length of the rotor is determined (104) by measuring one ormore characteristics of the rotorcraft with a sensor and comparing themeasured results with known characteristics of specific rotorcrafts.Once the specific type of rotorcraft is identified, the length of therotor can be determined by knowing the specifications of the rotorcraft.Example embodiments that measure characteristics of the rotorcraft todetermine actual length of the rotor are illustrated in FIGS. 3A and 3B.In the example embodiments, the characteristic measured is the frequencyof one or more rotors of the rotorcraft. Each specific rotorcraft hasits own rotor frequency (or rotor rate) during flight. The frequency isdependent, at least in part, on the size and weight of the craft and thenumber of blades that make up the rotor for the aircraft, which will bedifferent for each rotorcraft. Referring to FIG. 3A, a characteristicdetermination flow diagram 300 of one embodiment is illustrated. Thisembodiment implements an image sensor (such as the MWIR discussedabove). As illustrated, an image of the main rotor of a rotorcraft istaken (302). It is then determined if enough images have been taken todetermine the frequency (304). If there is not, another image is takenat step (302). In one embodiment, images are recorded over a selectperiod of time and at a select image rate to ensure enough images havebeen taken. In one embodiment, a single pixel in the images taken overthe select period of time is analyzed to determine the frequency of therotor. In particular, the temporal and spectral signature of the singleobserved pixel over the select period of time is analyzed. For example,referring to FIG. 4A, a temporal graph 400 illustrating a temporalsignal level of a single pixel over a period of 640 ms is illustrated.Moreover, a corresponding spectral signature graph 402 of the pixel isshown in FIG. 4B. The spectral signature graph 402 illustrates spectralsignal level vs. frequency. As illustrated, the frequency (or rotorrate), in this example, is about 20.51 Hz (first hump 403 of graph 402).FIG. 4B also illustrates a second hump 405, which is a first harmonic ofthe fundamental frequency. Periodic signals that are not perfectsinusoidal waves are composed of multiple sinusoidal waves offrequencies at multiple harmonics of the fundamental frequency. In thiscase, the fundamental frequency is about 20.5 Hz and the first harmonicfrequency is about 41 Hz. Once you determine the frequency (or rotorrate) (306), the measured frequency of the rotor is compared with knownfrequencies of rotorcrafts (308), as illustrated in FIG. 3A. In anembodiment, a frequency (rotor rate) of a known rotorcraft is stored ina table in memory 210 along with corresponding rotor lengths. If thereis a match (310), the length of the rotor is looked up in the table inmemory (312). In an embodiment, if there is not a match at (310), it isdetermined which is the closest frequency match (314) in the table andthe length associated with the closest frequency is then provided (316).

In the characteristic determination flow diagram 301 of FIG. 3B, notonly the frequency of a first main rotor is looked at to determine therotorcraft, but the frequency of a secondary rotor (such as a tail rotorof a helicopter) is determined. In this embodiment, a ratio rate betweenthe frequencies of the main rotor and the secondary rotor is used toidentify known rotorcraft and, hence, determine the actual length(diameter) of the main rotor. As with the first embodiment, thisembodiment takes images of the first main rotor of the rotorcraft (320).Once there are enough images taken (322), the frequency of the mainrotor is determined (324). As discussed above, in an embodiment, theimages are MWIR images and temporal and spectral signatures of a singleobserved pixel of the imaged main rotor over the select period of timeare analyzed to determine the frequency. Images are also taken of thesecondary rotor (326). Once enough images are taken (328), the frequencyof the secondary rotor is determined (330), as described above and shownin FIGS. 3A and 3B. In one embodiment, a single set of images is usedthat captures both the main rotor and the secondary rotor. With thisembodiment, a select pixel associated with the main rotor and a selectpixel associated with the secondary rotor are analyzed separately overtime to determine the different frequencies. Once the frequencies havebeen detected, a ratio rate between the frequencies is determined (332).The determined ratio rate is then compared with known ratio rates ofrotorcrafts (334). If there is a match (336), the actual length of themain rotor corresponding to the ratio rate is looked up in a table(338). In one embodiment, if there is not a match (336), the closestmatch of the ratio rate is determined (340) and the length of main rotorof the closest match is then looked up (342).

As discussed above in regard to FIG. 1, not only is the actual length(diameter) of the main rotor determined but also an angular extent ofthe main rotor is needed (106). Embodiments use the fact that theangular extent θ (shown in FIG. 6A) has a relationship that is dependenton the distance between the image sensor 212 and the rotorcraft 250. Thelarger the angular extent θ, the closer the rotorcraft 250 is to theimage sensor 212 of the home unit 202. An example angular extent flowdiagram 500 is illustrated in FIG. 5. As illustrated, the angular extentflow diagram 500 starts by taking images of the main rotor (502). In oneembodiment, an MWIR sensor is used to take the images. Moreover, in someembodiments the same images used to determine the angular extent areused to determine the frequency and actual distance of the main rotor asdiscussed above. The images are collected until the images, temporallyfiltered, trace out a complete ellipse (504). An ellipse is arepresentation of the main rotor from the perspective of the imagesensor. The major axis of the ellipse subtends the diameter of therotor, independent of the orientation of the rotor relative to thesensor 212, is located at (506). An example ellipse 600 with a majoraxis 602 is illustrated in FIG. 6A. In one embodiment, end points 604 aand 604 b of the major axis 602 are located (508) and used to determinethe angular extent θ (or subsumed angle) with a trigonometryrelationship. In the angular extent flow diagram 500 of FIG. 5, the endpoints 604 a and 604 b of FIG. 6A are first converted to unit lookvectors (510) using knowledge of lens distortion of the lens 211 (shownin FIG. 6A) of the image sensor 210. In some embodiments, for each pixelin the image there is going to be a mapping to a vector that points to aparticular direction. So, each end point maps to a vector associated toeach respective pixel. The angular extent is determined (512) as theinverse cosine of the inner product (or “dot” product) of the vectors inthis example. Given two vectors (x1, y1, z1) and (x2, y2, z2), the innerproduct is the number x1×x2+y1×y2+z1×z2.

In particular, in this embodiment, each pixel in an image, whether it isformed using a consumer digital camera or a military-grade infraredsensor, corresponds to a specific point in three-dimensional space.Therefore, each pixel corresponds to a vector that originates at thesensor and points in the direction of the object represented by thatpixel. The distance to the object is unknown; however, so only thedirection of the vector is known. Since each pixel corresponds to adifferent part of the 3D scene, each pixel corresponds to a uniquepointing vector. The mapping from pixel location to pointing vector maybe determined during the design process or by measurement. Eachcomponent of the sensor, including the lens, affects the mapping.Assuming the sensor is rigid, the determination of this mapping is aone-time process. Given two pixel locations (C₁, R₁) and (C₂, R₂) (C andR indicate column and row, respectively), two corresponding unit vectorsU₁ and U₂ are determined based on this mapping. The angle between thesevectors, θ, is calculated using θ=cos⁻¹(U₁×U₂). This is illustrated inFIG. 6B.

In yet another embodiment, the determination of the actual distance ofthe main rotor is determined using one of the techniques to determinethe measured distance of the major axis main rotor discussed in relationto FIGS. 5 and 6A and to also determine the measured major axis of thesecondary rotor. A determined ratio of the major axis of the main rotorto the major axis of the secondary rotor is then used to compare withknown ratios of main rotors to secondary rotors of rotorcraft todetermine the actual distance of the rotor (diameter), similar to thematching process of FIGS. 3A and 3B above.

As briefly discussed above in regard to FIG. 1, once you have the actuallength of the main rotor and the angular extent one can determine thedistance between the home unit 202 and rotorcraft 250 (108). In oneembodiment, the distance or range between the home unit 202 and therotorcraft 250 is determined by the following equation:

${L = {\frac{}{2\; \tan^{\frac{\theta}{2}}} \approx {{/\theta}}}},$

where L is the distance or range between the home unit and therotorcraft, d is the actual diameter of the rotor and θ is the angularextent (subsumed angle). In some embodiments, the controller 204implements the equation to determine the distance (range) between thehome unit 202 and the rotorcraft 250. Moreover, as discussed above, insome embodiments, the distance of the rotorcraft is monitored over timeuntil it is outside the range of the sensor. This is done bycontinuously capturing image sets (that provide the then current angularextent) with the sensor and monitoring the change in the angular extentover time.

Although specific embodiments have been illustrated and describedherein, it will be appreciated by those of ordinary skill in the artthat any arrangement, which is calculated to achieve the same purpose,may be substituted for the specific embodiments shown. This applicationis intended to cover any adaptations or variations of the presentinvention. Therefore, it is manifestly intended that this invention belimited only by the claims and the equivalents thereof.

What is claimed is:
 1. A method, comprising: capturing sensor data by asensor of a home unit, the sensor data including one or morecharacteristics of a rotor of a rotorcraft; determining an actual lengthof the rotor of the rotorcraft from sensor data; determining an angularextent of a major axis of the rotor from a perspective of the home unit;and determining a distance from the home unit to the rotorcraft based,at least in part, on the determined actual length of the rotor and theangular extent.
 2. The method of claim 1, wherein determining the actuallength of the rotor of the rotorcraft is based, at least in part, on adetermined frequency of the rotor from analyzing the sensor data.
 3. Themethod of claim 2, wherein analyzing the sensor data includes analyzingat least one of a temporal signature or spectral signature of a selectpixel over a series of images captured by the home unit.
 4. The methodof claim 2, wherein determining the actual length of the rotor of therotorcraft includes comparing the determined frequency with a lookuptable having known frequencies corresponding to known rotor lengths. 5.The method of claim 2, wherein determining the actual length of therotor of the rotorcraft includes: determining another frequency foranother rotor of the rotorcraft; determining a ratio for the frequencyof the rotor and the another frequency of the another rotor; andcomparing the ratio with a lookup table having known ratioscorresponding to known rotor lengths of rotorcrafts having multiplerotors.
 6. The method of claim 5, further comprising the home unitanalyzing a same set of images for determining both the determinedfrequency of the rotor and the another frequency of the another rotor.7. The method of claim 1, further comprising the home unit analyzing asame set of images for determining the actual length of the rotor of therotorcraft and determining the angular extent of the major axis of therotor of the rotorcraft.
 8. The method of claim 1, wherein the distancefrom the home unit to the rotorcraft is approximately the actual lengthof the rotor divided by the angular extent determined from the sensordata.
 9. The method of claim 1, further comprising determining adirection of travel for the rotorcraft by comparing the determineddistance of the rotorcraft from at least two different points in time.10. The method of claim 9, further comprising determining whether therotorcraft and the home unit are on a collision path based, at least inpart, on the determined distance and the direction of travel for therotorcraft.
 11. The method of claim 10, further comprising generating analarm responsive to a determination that the rotorcraft and the homeunit are on a collision path.
 12. A method, comprising: determining adistance from a home unit to a nearby rotorcraft based, at least inpart, on comparing at least one measured characteristic of a capturedimage of the nearby rotorcraft with corresponding known characteristicsof known rotorcrafts; determining a direction of travel for the nearbyrotorcraft; and generating an alarm if a collision course is determinedfor the home unit and the nearby rotorcraft.
 13. The method of claim 12,wherein the at least one measured characteristic and corresponding knowncharacteristics include an actual length of a rotor, and an angularextent of a major axis of the rotor from a perspective of the home unit.14. A rotorcraft collision avoidance system for a home unit, comprising:at least one imaging sensor configured and arranged to capture images ofa nearby rotorcraft; and a controller operably coupled with the at leastone imaging sensor, the controller configured to: receive the capturedimages from the at least one imaging sensor; and process the capturedimages to: identify a rotor of the nearby rotorcraft; and determinedistance to the nearby rotorcraft based, at least in part, on adetermined actual length of the rotor and a determined angular extent ofa major axis of the rotor from a perspective of the home unit.
 15. Therotorcraft collision avoidance system of claim 14, wherein thecontroller is further configured to determine the major axis of therotor by temporally filtering the plurality of images to obtain anellipse image of the rotor, and identifying a longest measurement acrossthe ellipse image.
 16. The rotorcraft collision avoidance system ofclaim 15, wherein the controller is further configured to determine theangular extent by locating pixels that correspond to end points of themajor axis, converting the end point pixels to vectors, and determiningan inverse cosine of an inner product of the vectors.
 17. The rotorcraftcollision avoidance system of claim 15, wherein the controller isfurther configured to determine the angular extent of the major axis ofthe rotor by counting a number of pixels along the major axis of theellipse image to determine a pixel space, and applying a linearrelationship to the pixel space.
 18. The rotorcraft collision avoidancesystem of claim 14, further comprising a memory having at least onelookup table stored thereon with known frequencies corresponding toknown actual rotor lengths used by the controller to determine theactual length of the rotor.
 19. The rotorcraft collision avoidancesystem of claim 14, wherein the at least one imaging sensor includes atleast one of a mid-wave infrared imaging sensor, a radar system, anacoustic system, a digital camera, or an infrared sensor;
 20. Therotorcraft collision avoidance system of claim 14, wherein the home unitis selected from the group consisting of a rotorcraft, an aircraft, avehicle, or a stationary structure.